Neuro-symbolic AI can be considered as a pathway to achieve artificial general intelligence by integrating knowledge representation (KR) and machine learning (ML) and leading to improvements in scalability, efficiency, and explainability. In this project, we plan to explore the use of neuro-symbolic AI to implement a natural language processing (NLP) component, which is able to comprehend and answer queries related to maps (as used for autonomous driving) formulated in natural language.

The topic is subdivided in two complementary master theses:

Thesis 1: Ontology-assisted automatic parsing of NDS map BLOBs

The aim of the first thesis, from a functional point of view, is to transform an opaque NDS map BLOB into a knowledge graph model that can be queried and used. In advanced driver assistant systems (ADAS), map data is usually coded as binary blocks to minimize transfer and storage costs, which implies parsing of the related BLOBs to be able to use the encapsulated data. Due to the complexity and ambiguity of the navigation data standards (NDS), this parsing step causes development and maintenance difficulties.
In this thesis, we would like to instigate a data-driven approach to parse the map binary data and generate the correspondent RDF graph. In fact, the map BLOB can be considered as a picture with different objects to be recognized and therefore, a computer vision system based on Deep Neural Networks (DNN) can be used to identify those objects. In order to enhance the efficiency of this approach, a knowledge graph embedding (KGE) method is to be used to incorporate the semantics of the map data, represented by its ontology, into the ML task.

Thesis 2: Natural Language Processing (NLP) for map data

The second thesis aims to use the KG generated in the first thesis and train a Large Language Model (LLM) to transform map related natural human questions (e.g., what is the speed limit at some GPS point? How many lanes do we have at some GPS point or at certain address? How far is the closest pharmacy? Etc.) into SAPRQL queries and execute them to answer those questions.
KGE can be also used here to enhance the comprehension of the human language questions by the LLM system and generate the correct query. In addition, symbolic reasoning on the generated RDF graph shall be used to check the consistency and the correctness of the provided answers. Another envisioned advantage of using knowledge in language models is that they get the additional information through entities or KG triples that otherwise require a lot of data to learn.

For further information, please contact Birte Glimm or consult the BMW offer with possibility to apply.